172 research outputs found
Low-Rank Matrix Approximation with Weights or Missing Data is NP-hard
Weighted low-rank approximation (WLRA), a dimensionality reduction technique
for data analysis, has been successfully used in several applications, such as
in collaborative filtering to design recommender systems or in computer vision
to recover structure from motion. In this paper, we study the computational
complexity of WLRA and prove that it is NP-hard to find an approximate
solution, even when a rank-one approximation is sought. Our proofs are based on
a reduction from the maximum-edge biclique problem, and apply to strictly
positive weights as well as binary weights (the latter corresponding to
low-rank matrix approximation with missing data).Comment: Proof of Lemma 4 (Lemma 3 in v1) has been corrected. Some remarks and
comments have been added. Accepted in SIAM Journal on Matrix Analysis and
Application
Spectrum optimization in multi-user multi-carrier systems with iterative convex and nonconvex approximation methods
Several practical multi-user multi-carrier communication systems are
characterized by a multi-carrier interference channel system model where the
interference is treated as noise. For these systems, spectrum optimization is a
promising means to mitigate interference. This however corresponds to a
challenging nonconvex optimization problem. Existing iterative convex
approximation (ICA) methods consist in solving a series of improving convex
approximations and are typically implemented in a per-user iterative approach.
However they do not take this typical iterative implementation into account in
their design. This paper proposes a novel class of iterative approximation
methods that focuses explicitly on the per-user iterative implementation, which
allows to relax the problem significantly, dropping joint convexity and even
convexity requirements for the approximations. A systematic design framework is
proposed to construct instances of this novel class, where several new
iterative approximation methods are developed with improved per-user convex and
nonconvex approximations that are both tighter and simpler to solve (in
closed-form). As a result, these novel methods display a much faster
convergence speed and require a significantly lower computational cost.
Furthermore, a majority of the proposed methods can tackle the issue of getting
stuck in bad locally optimal solutions, and hence improve solution quality
compared to existing ICA methods.Comment: 33 pages, 7 figures. This work has been submitted for possible
publicatio
On the geometric interpretation of the nonnegative rank
The nonnegative rank of a nonnegative matrix is the minimum number of nonnegative rank-one factors needed to reconstruct it exactly. The problem of determining this rank and computing the corresponding nonnegative factors is difficult; however it has many potential applications, e.g., in data mining, graph theory and computational geometry. In particular, it can be used to characterize the minimal size of any extended reformulation of a given combinatorial optimization program. In this paper, we introduce and study a related quantity, called the restricted nonnegative rank. We show that computing this quantity is equivalent to a problem in polyhedral combinatorics, and fully characterize its computational complexity. This in turn sheds new light on the nonnegative rank problem, and in particular allows us to provide new improved lower bounds based on its geometric interpretation. We apply these results to slack matrices and linear Euclidean distance matrices and obtain counter-examples to two conjectures of Beasly and Laffey, namely we show that the nonnegative rank of linear Euclidean distance matrices is not necessarily equal to their dimension, and that the rank of a matrix is not always greater than the nonnegative rank of its square.nonnegative rank, restricted nonnegative rank, nested polytopes, computational complexity, computational geometry, extended formulations, linear Euclidean distance matrices.
A multilevel approach for nonnegative matrix factorization
Nonnegative Matrix Factorization (NMF) is the problem of approximating a nonnegative matrix with the product of two low-rank nonnegative matrices and has been shown to be particularly useful in many applications, e.g., in text mining, image processing, computational biology, etc. In this paper, we explain how algorithms for NMF can be embedded into the framework of multi- level methods in order to accelerate their convergence. This technique can be applied in situations where data admit a good approximate representation in a lower dimensional space through linear transformations preserving nonnegativity. A simple multilevel strategy is described and is experi- mentally shown to speed up significantly three popular NMF algorithms (alternating nonnegative least squares, multiplicative updates and hierarchical alternating least squares) on several standard image datasets.nonnegative matrix factorization, algorithms, multigrid and multilevel methods, image processing
Nonnegative factorization and the maximum edge biclique problem
Nonnegative matrix factorization (NMF) is a data analysis technique based on the approximation of a nonnegative matrix with a product of two nonnegative factors, which allows compression and interpretation of nonnegative data. In this paper, we study the case of rank-one factorization and show that when the matrix to be factored is not required to be nonnegative, the corresponding problem (R1NF) becomes NP-hard. This sheds new light on the complexity of NMF since any algorithm for fixed-rank NMF must be able to solve at least implicitly such rank-one subproblems. Our proof relies on a reduction of the maximum edge biclique problem to R1NF. We also link stationary points of R1NF to feasible solutions of the biclique problem, which allows us to design a new type of biclique finding algorithm based on the application of a block-coordinate descent scheme to R1NF. We show that this algorithm, whose algorithmic complexity per iteration is proportional to the number of edges in the graph, is guaranteed to converge to a biclique and that it performs competitively with existing methods on random graphs and text mining datasets.nonnegative matrix factorization, rank-one factorization, maximum edge biclique problem, algorithmic complexity, biclique finding algorithm
Using underapproximations for sparse nonnegative matrix factorization
Nonnegative Matrix Factorization (NMF) has gathered a lot of attention in the last decade and has been successfully applied in numerous applications. It consists in the factorization of a nonnegative matrix by the product of two low-rank nonnegative matrices:. MªVW. In this paper, we attempt to solve NMF problems in a recursive way. In order to do that, we introduce a new variant called Nonnegative Matrix Underapproximation (NMU) by adding the upper bound constraint VW£M. Besides enabling a recursive procedure for NMF, these inequalities make NMU particularly well suited to achieve a sparse representation, improving the part-based decomposition. Although NMU is NP-hard (which we prove using its equivalence with the maximum edge biclique problem in bipartite graphs), we present two approaches to solve it: a method based on convex reformulations and a method based on Lagrangian relaxation. Finally, we provide some encouraging numerical results for image processing applications.nonnegative matrix factorization, underapproximation, maximum edge biclique problem, sparsity, image processing
Low-rank matrix approximation with weights or missing data is NP-hard
Weighted low-rank approximation (WLRA), a dimensionality reduction technique for data analysis, has been successfully used in several applications, such as in collaborative filtering to design recommender systems or in computer vision to recover structure from motion. In this paper, we study the computational complexity of WLRA and prove that it is NP-hard to find an approximate solution, even when a rank-one approximation is sought. Our proofs are based on a reduction from the maximum-edge biclique problem, and apply to strictly positive weights as well as binary weights (the latter corresponding to low-rank matrix approximation with missing data).low-rank matrix approximation, weighted low-rank approximation, missing data, matrix completion with noise, PCA with missing data, computational complexity, maximum-edge biclique problem
An interior-point method for the single-facility location problem with mixed norms using a conic formulation
We consider the single-facility location problem with mixed norms, i.e. the problem of minimizing the sum of the distances from a point to a set of fixed points in R, where each distance can be measured according to a different p-norm.We show how this problem can be expressed into a structured conic format by decomposing the nonlinear components of the objective into a series of constraints involving three-dimensional cones. Using the availability of a self-concordant barrier for these cones, we present a polynomial-time algorithm (a long-step path-following interior-point scheme) to solve the problem up to a given accuracy. Finally, we report computational results for this algorithm and compare with standard nonlinear optimization solvers applied to this problem.nonsymmetric conic optimization, conic reformulation, convex optimization, sum of norm minimization, single-facility location problems, interior-point methods
Algorithms for Positive Semidefinite Factorization
This paper considers the problem of positive semidefinite factorization (PSD
factorization), a generalization of exact nonnegative matrix factorization.
Given an -by- nonnegative matrix and an integer , the PSD
factorization problem consists in finding, if possible, symmetric -by-
positive semidefinite matrices and such
that for , and . PSD
factorization is NP-hard. In this work, we introduce several local optimization
schemes to tackle this problem: a fast projected gradient method and two
algorithms based on the coordinate descent framework. The main application of
PSD factorization is the computation of semidefinite extensions, that is, the
representations of polyhedrons as projections of spectrahedra, for which the
matrix to be factorized is the slack matrix of the polyhedron. We compare the
performance of our algorithms on this class of problems. In particular, we
compute the PSD extensions of size for the
regular -gons when , and . We also show how to generalize our
algorithms to compute the square root rank (which is the size of the factors in
a PSD factorization where all factor matrices and have rank one)
and completely PSD factorizations (which is the special case where the input
matrix is symmetric and equality is required for all ).Comment: 21 pages, 3 figures, 3 table
Real-time dynamic spectrum management for multi-user multi-carrier communication systems
Dynamic spectrum management is recognized as a key technique to tackle
interference in multi-user multi-carrier communication systems and networks.
However existing dynamic spectrum management algorithms may not be suitable
when the available computation time and compute power are limited, i.e., when a
very fast responsiveness is required. In this paper, we present a new paradigm,
theory and algorithm for real-time dynamic spectrum management (RT-DSM) under
tight real-time constraints. Specifically, a RT-DSM algorithm can be stopped at
any point in time while guaranteeing a feasible and improved solution. This is
enabled by the introduction of a novel difference-of-variables (DoV)
transformation and problem reformulation, for which a primal coordinate ascent
approach is proposed with exact line search via a logarithmicly scaled grid
search. The concrete proposed algorithm is referred to as iterative power
difference balancing (IPDB). Simulations for different realistic wireline and
wireless interference limited systems demonstrate its good performance, low
complexity and wide applicability under different configurations.Comment: 14 pages, 9 figures. This work has been submitted to the IEEE for
possible publicatio
- …